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ABSTRACT 

In an effort to improve the reliability of the 
Noncognitive Questionnaire (NCQ) , the instrument was revised and the 
revision's reliability and validity were examined examined with black 
and white college freshmen. The NCQ measures eight non-traditional or 
non-cognitive variables that relate to irinority student retention. 
The revised NCQ is composed of 67 items; 38 Likert type items were 
added to the NCQ to create the revised form. Initially, the fit of 
the revised NCQ to eight hypothesized constructs was examined using 
confirmatory factor analysis on a sample of lOl blade students. It 
was found that the revised instrument adequately represented the 
data. Tests of the invariance of the factor structure obtained on the 
initial black samples compared to a second black sample of 97 
students and a sample of 202 white students revealed that the factor 
structure held across samplea. It was concluded that the revised 
instrument was content valid and that the scales were stable and 
invariant across race. Five data tables are presented. (TJH) 
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Summary 

The predictive validity of the Non-Cognitive Questinnaire (Tracey 
& Sedlacek, 1984) has been demonstrated in past research, however 
some of the scries of this instrument have been found to have 
relatively low reliabilities. In an effort to improve 
measurement, the NCQ was revised and the reliability and validity 
examined on black and white samples of college entrants. First 
the fit of the revised NCQ to the eight hypothesized constructs 
was examined using confirmatory factor analysis on a sample of 
101 black students^ It was found that the revised instrument 
adequately represented the data. Tests of the invariance of the 
factor structure obtained on the initial black sample to a second 
black sample (N=97) and a white sample (N=202) revealed that the 
factor structure held across samples. It was concluded that the 
NCQ-R was content valid and that the scales were stable and 
invariant across race* 
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Factor Structure of the Non-Cognitive Questionnaire-Revised 
Across Samples of Black and White College Students 

There is a growing awareness that academic success is a 
function of more than just academic ability. It has been 
demonstrated that other less "intellectual" dimensions are as or 
more important in accounting ror academic success in higher 
education (Pantages & Creedcn. 1978). Much of this research 
supports Tinto's (1975) contention that the academic and social 
integration into the institution are key determinants of academic 
success, especially persistence (Aitken, 1982; Bean, 1980; Munro, 
198-1; Pascarella & Chapman, 1983). 

Although there is evidence that noncognitive dimensions are 
important determinants of academic success for all students 
lAstin, 1975; Messick, 1979; Nelson. Scott, & Bryan, 1984; 
Stoecker, Pascarella. k Woltle. 1988). these less traditional 
measures appear to be especially important predictors of minority 
academic success in higher education (Astin. 1982; Fleming, 1984; 
Nettles, Thoeny, & Gosman, 1986; Sedlacek, 1988; Wolfle, 1985). 
Reviewing the research in this area, Sedlacek and Brooks (1976) 
posited eight noncognitive dimensions that were important in 
minority student academic success. These eight dimensions were: 
positive self -concept, realistic self -appraisal, understanding of 
and and ability to deed with racism, preference for long range 
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goals over more iimiediate short-term needs, support of others for 
academic plans, successful leadership experience, demonstrated 
community service, and academic interest and familiarity. 

To assess these eight dimensions. Tracey and Sedlacek (1984), 
developed the Non-Cognitive Questionnaire (NCQ). It has been shown 
in subsequent research that the NCQ was: (a) content valid, i.e., 
that the instrument items loaded on the hypothesized general 
dimensions (Tracey & Sedlacek, im), (b) predictive of grades 
over four years for both black and white students (Tracey & 
Sedlacek, 1984, 1985), ,c) highly predictive of persistence 
esRecially for black students above that obtained using academic 
aptitude measures (Tracey & Sedlacek. 1985), (d) highly predictive 
of eventual graduation, especially for black students (Tracey & 
Sedlacek, 1987a), (e) predictive of persistence for specially 
admitted students iWhite & Sedlacek, 1986;. and predictive of 
grades and persistence for international students (Boyer ic 
Sedlacek, 1988). However Tracey and Sedlacek (1987b) found that 
some of the scales were not as strong psychometricly as desired. 
Specifically some of the internal consistency estimates were low. 

The purpose of this study was to refine the instrument such 
that some of the subscales would more reliably ana accurately 
reflect the desired construct. To this end the number of items 
was increased. The reliability and validity of this revision was 
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examined through the application of confirmatory factor analysis. 
Specifically the fit of this revision to the proposed eight 
dimension model was examined. In order to further support the 
revision the invariance of the factor structure of the 
Non-Cognitivs Questionnaire-Revised (NCQ-R) across racial groups. 

Method 

Sample 

a sample of new freshmen students attending summer orientation 
at .a large, eastern, state university were g .ven the NCQ-R during 
the summer of 1987. Summer orientation it attended by more than 
90% of the new freshmen each year. 

Students attending summer orientation typically are required to 
fill out several instruments in a group assessment. The specific 
groups of students that were given the NCQ-R were randomly 
selected. NCQ-R instruments were completed by a total of 1633 
students. This group contained 198 blacks, 1269 whites and 166 
students of other ethnic/ cultural background, or those who did not 
identify themselves. Only the black and white samples were 
examined . 

Instruments 



i 
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Non-Cosniiive Questionnaire-Rgj^j^ (ncQ-R) is an extended 
version of the Non-Cognitive Questionnaire (Tracey & Sedlacek. 
1984). The NCQ-R is composed of 67 items and is identical to the 
NCQ except 38 more Likert type items were added to gain a better 
assessment of the eight noncognitive dimensions proposed by 
Sedlacek and Brooks (1976) to be related to minority academic 
success in higher education. The NCQ-R is composed of 58 
statements pertaining to perceptions and expectations of one's 
academic career (responded to using a 5 point Likert type fonr.at, 
l=strongly agree. 5=strongly disagree), two items concerning the 
amo.unt of education expected, three items requesting that the 
student list current goals, accomplishments, and outside 
activities, and several background items (gender, sex. race, and 
age). The 67 items comprised two sets, one set which was designed 
to directly isflect the noncognitive dimensions (38 items) and a 
second set of experimental items. The first set of items, i.e. 
those directly relating to the eight noncognitive dimensions were 
examined ih this study. Each of the eight dimensions was 
represented by 3 to 7 items. 
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This study was designed to assess the validity of the factor 
structure of the NCQ-R. Specifically, the validity of the eight 
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subscale structure of the NCQ-R was examined using Confirmatory 
Factor Analysis (CFA) on the item covariance matrix through the 
LISREL VI (Joreskog u Sorbom, 1986) package. Since the factor 
structure of the NCQ-R had yet to be established, it was initially 
appropriate to estimate the loadings of this instrument. Since 
primary interest was in the validity of the instrument with 
minority students, the black sample v/as the one where this initial 
estimation was examined. Given that 38 separate items were 
included on the subscaies and to reduce computer costs which can 
be high using LISREL on a data set of this size, items were summed 
to create two or three item subscaies. A listing of the items 
included in subscaies is presented in Table 1. These 24 subscaies 
were then used as the indicator variables of the eight 
noncognitive dimensions. 



Insert Table 1 About Here 



However, the factor loadings obtained in one sample may not 
reflect those obtained in another because of the inclusion of 
sample idiosyncratic covariance. So it was important to obtain an 
independent estimate of the validity of these factor loading 
estimates on another sample of black students. This was done by 
randomly splitting the black sample into two subsamples, one for 
parameter estimation and the second to test the generalizability 
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of these parameters. The analysis in this study consiste^d of 
three steps. First, half of the black sample (N=10l) was used to 
estimate the factor loading parameters. Then the invariance of 
these parameters in the second black sample (N=97) was examined. 
Finally, the invariance of these parameter estimates across race 
was examined by testing the model derived on the first black 
sample on a random sample of the white students (N=222). 



Results 



Prior to ail analyses, the internal consistency of each of the 
eight dimensions was estimated for each of the three samples. 
These reliability estimates are listed in Table 2. This enabled an 
initial check on the appropriateness of using smaller 2 and 3 item 
subscales instead of individual items in the subsequent CFA. 



Insert Table 2 About Here 

The test of the fit of the proposed factor structure to the 
first black student sample was found to be adequate. Although che 
maximum likelihood goodness of fit chi-square was significant 
2 

a <N=101,181)=250.24, £<.001), the chi- square/degrees of freedom 
ratio (1.38) was well within good fit range. Marsh and Hocevar 
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(1985) suggest chi-square/degrees of freedom ratios below wo are 
indicative of adequate fit. Father indications of a good fit 
were the GFI (goodness of fit index) of .83. a root mean square 
residual of .042, a modification index matrix of uniformly low 
values (six values greater than 5.0) and a nomalized residual 
matrix with values below 2.0. The Tucker-Lewis Index (TLI, Tucker 
& Lewis, 1973) was also calculated because Marsn, Balla, and 
McDonald (1968) have found that this is one of the few indices of 
fit that is independent of sample size. The TLI for this model on 
the black sample was .85. The resulting standardized parameter 
estimates of the factor loadings (Lambda X) were all found to be 
significant (ratio of parameter estimate to the standard error was 
greater than 2.0) and are presented in Table 3. The standardized 
parameter estimates of the relations a.^onp, the eight dimensions 
(phi matrix) are presented in Table li. 



Insert Table 3 About Here 



Insert Table 4 About Here 



The invariance of this model was examined with respect to the 
second black sample and a sample of white students. This 
invariance was examined in a sequential stepwise process, first, 
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the equality of item covariance matrices was examined. The tests 
of this equality are presented in Table 5. 

Insert Table 5 About Here 



Although the chi-squared statistic of the equal covariance 
between the first and second black samples was significant, the 
chi-square/degrees of freedom ratio of ^,96 was below 2.0. This 
result led us to conclude that the covariance matrices were not 
different. This conclusion was further supported by the 
relatively good fit indicators obtained when the model derived on 
the first black sample was applied to the second black sample (see 
Table 5). 

The examination of the equality of the covariance matrices of 
the original black student sample with the sample of white 
students also yielded similar results. Although the chi-square 

2 

result was significant (X (N=323,253)=384.37, 2<.001), the 

chi- squared/degrees of freedom ratio was below 2.0, and the tit of 

the black model to the white data was generally adequate. 

Discussion 

Overall, the results are supportive of the validity of the 
NCQ-R measuring the eight noncognitive dimensions posited by 

.52 
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for the white sample on the racism subscale. This makes intuitive 
sense given that this dimension should be less salient for whites 
it is for blacks. Items reflecting interacting with other 
cultures and relative comfort level in a predominately white 
university are measuring different dijr.ensions for white students. 
For blacks these items are assessing more of the same diiP.ension. 
Thus the validity of this one scale for whites is questionable, 
although it is likely that those items that concern "negotiating 
the system" are valid for all students. 

An examination of the correlations among the eight constructs 
reveals that there is a fair amount of overlap especially 
involving the constructs of racism and realistic self-appraisal. 
This coiTjnunality could be somewhat attributable tc the difficulty 
in defining these constructs. But, in addition this could be due 
to the importance of each of these in relation to the other six 
noncognitive dimensions. There may be a higher-order construct 
accounting for these relations. Or perhaps these dimensions of 
racism and realistic self -appraisal are the crucial aspects that 
are being assessed in the NCQ-R. It was demonstrated in predictive 
validity studies of the original NCQ (Tracey & Sedlacek, lb-84, 
1985, 1987a, 1987b) that these two dimensions were especially 
important in predicting academic success for black students. 

Finally, there was adequate support found for the invariance of 
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the factor structure of the NCQ-R across samples of black students 
as well as across race. The validation sample of blacks were 
found to respond to the NCQ-R in a similar manner as the original 
black sample. The white sample was also found to have a similar 
response pattern as the black sample. So the Non-Cognitive 
Questionnaire- Revised has been found to have adequate support for 
application with white and black students. 

There are several next steps that are required in the 
development and application of the instrument. Certainly, the 
validity of the instrument needs to be examined at other 
institutions. Though the support for it is good with these 
samples examined here, they were ail generated at one university. 
More examination of the generalizabiiity of the factor structure 
is required. Also, though the original NCQ has been found to have 
good predictive validity, especially with respect to blaoc student 
persistence and eventual graduation (Tracey & Sedlacek, 1985, 
1987a), the i^redictive validity of the revision needs 
examination. However, given that many of the same items form part 
of ths subscales in the NCQ-R, it is expected that there would be 
equally high predictive validity for the NCQ-R. 

Given the very different pattern of academic success between 
black and white students and the very different entry and 
attrition rates (Rstin, 1982; Sedlacek & Pelham, 1976; Sedlacek 
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Webster, 1978) it is Liiportant that attempts be made to determine 
the dimensions related to these differences and ameliorate them. 
The eight dimensions proposed by Sediacek and Brooks are a start 
in this area. The NCQ-R appears to be a useful instrument to 
examine student success in higher education. 
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Table 1 

Items Included on Each of the Eight Noncognitive Diinensions and 
the Separate Indicator Subscales. 



^^^^ Indicator Subscale 



academic Positive Sgif-eonceot {PSO 



My high school grades don't really reflect what I can do. PSCl 

T^^^r ^ ^^'"^ n^ost students here. psci 

It should not be very hard to get a B average here. PSC2 

I am as skilled academically as the average applicant here. P.SC2 



Realistic Self Appraisal <RSa) 

I know the areas where I am weak and X try to iinprove them. 
I want a chance zo prove myself academically. 
If tutoring IS made available on campus at no cost, I would 
attend regularly, ^ 
When I believe strongly on something I act on it. Rsa3 
I try to find opportunities to learn new things. rsa3 

Support of Academic Plans (SUP) 

My friends and relatives don't feel I should go to college.^ SUPl 

My family has always wanted me to go to college. SUP2 
If I run into problems concerning school. I have someone 

who would listen to me and help me. sUP3 



Rsni 



LEAl 
LEP.1 



Leadership (LER) 

I am sometimes looked up to bv others 
In groups where .1 am comfortable. I m often looked 
to as leader. 

I was a leader in high school. .pn-j 
My friends look at me to mak-j decisions. 
I am not good at getting others to go along with me.* LEA3 

LQM nariixe Goals (LRG) 

I prefer to be spontaneous rather than to make plans.* lRGI 
I usually mark important dates on my calendar. LRGl 
I know what I want to be doing 10 years from now. LRG2 

(Table 1 continued on next page) 
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Table 1 continued 



I have studied ^jliings about my major field on my own"^ LR6^ 

I often make lists of things to do. tj^g2 
I have talked about my career goals with someone who 

has worked in that field. LRG3 
I have already learned something in my proposed maior 

outside of high school. y^g3 



iunitv Ties (COM) 

I expect to be involved in many off-campus activities 

while enrolled here. 
I don't expect to get co know faculty personally 

during my first year.* 
I am comfortable interacting with people from other races 

or cultures. 

I e;tpect the faculty to treat me differei^tly from the 

average student here. 
I enjoy working with others. 
I keep to myself pretty much.* 

I find I get more comfortable in a new place as soon as I 
■ make some good friends, 

I expect to find lots of people who are lik« me here. 
Understanding of Racism (RJIC) 

I expect to have little contact with students from 
other races. 

My friends are exclusively the same race as I am. 
My background should help me fit in well here.* 
It should not be very hard to get a B average here. 

Academic Familiar ity (RCF) 



COMl 

COHl 

COM2 

C0H2 

C0K3 
C0K3 

com 
com 



rjici 

RACl 
RAC2 

mc2 



I have studied things abont my major field on my own. ACFl 
I have talked about my career goal with someone who works 

m that career. ACFl 

I have learned more outside of school than in school . ACF2 

Degree of academic reiatedness of three current goals. ACF2 



•* indicates that the item was reverse coded. 
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Table 2 

Estiiiiates of Internal Consistencies (Alpha) of Each Non-Cognitive 
Dimension by Sample. 



Dimension 



academic Positive 
Self-Concept 

Realistic Self 
Appraisal 

Support of Academic 
Plans 

Leadership 

Long Range Goals 

Community Ties 

Racism 

Academic Familiarity 



n items 



Black I 



.60 

.58 

.84 
.79 
.65 
.61 

,55 
6b 



Sample 



Black 2 



.49 

.56 

.53 
.83 
.72 
,57 
.54 
.74 



White 



.40 

.49 

.49 
.82 
.68 
.70 
.37 
.60 



Table 3 
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Standardized Parameter Estjji,at«s (LanO^da X) of: the KCG-Xi Generated 
on the First Black Sample. 



,30 
,37 
,26 



Dimension 

Variable PSC Rsa sUP LEA LliG COM R7.C m 

PSCl .26 
PSC2 -.92 
RSAl 

RSa2 .42 
RSA3 47 
SUPl 
SUP2 
SUP3 
LEAl 

Lm.-3 

LRGl 
LRG2 
LRG3 
OOMl 

com 

COM3 
COM4 

MC2 • -''i 

ACFl '^^J- 
ACF2 . 82 

.51 



,';9 
,56 
,30 

,19 
.41 
.87 

.30 
-,2li 
.61 
,19 
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Table ^^ 

Standardised Parameter Esti^nates of the Relations nmong the Eight 
Non-Cognitive Dimensions (PHI) 



Dimension 



Dimension PSC RSA SUP LEfi LRG COM RflC ?.CF 



PSC 


1.00* 










RSk 


-.19 


1.00* 








SUP 


.20 


-.3fe* 


1.00* 






LEA 


-.22 


.33* 


-.22 


i.OO* 




LRG 


-.04 


.52* 


-.36 


.29 


1.00« 


cm 


-.11 


,62* 


-.47* 






.67* 
.02 


-.38* 


.37* 
-.37* 


- . 32* 
.26 


-.17 
.15 



1.00* 

-.5.V* 1.00* 
.51* -.11 i.00=» 



':p<.05 b3.<>ed on the ratio of the ratio of parameter of- the phi 
matrix to its standard error. 
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Table 5 

Goodness of Fit Indices for the Comparison of Factor Inv:<riance 



Samples Compared 



Black 1 vs. Black 2 Black 1 vs. White 



Test of Equality of Covariance Matrices 

'Kaximuh. .ik.elihood 

cM-squared 496 . 97 35,^ ^ 37 

Chi-squared/df ratio i.gg j 52 

Goodness of fit of Second Sample Using Model Obtained on First Sample 

Goodness of Fit Index 

6FI Scunple 2 .77 .j,^ 

Root Mean Scjwared Residual 

im. Saiiple 2 .071 ,0^,5 

Tucker-Lewis index 

T)'jJ. Sample 2 .73 
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